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* 

The  need  for  considering  cost  in  relation  to  effectiveness  4  *” 
must  have  occurred  to  the  earliest  planners.  Cost-effectiveness 
is  not  a  catchword  to  suggest  we  are  doing  something  new;  at  i 

most  we  are  doing  something  better.  What  is  novel  about  cost- 
effectiveness  analysis  today,  and  what  brings  us  together,  is 
the  marvelous  refinement  of  the  methods  for  relating  cost  to 
effectiveness  that  has  taken  place  in  the  last  few  years  and 
the  acceptance  of  these  methods  as  an  aid  to  decisionmaking 
at  high  policy  levels.  By  this  Symposium  we  hope  to  make  clear 
the  scope  of  cost-effectiveness  analysis  and  to  discuss  its 
problems  and  limitations  in  order  that  we  may  broaden  its 
scope  and  increase  its  acceptance. 


DEFINITIONS 

What  is  a  cost-effectiveness  analysis?  Broadly  defined 
(too  broadly  for  my  taste)  it  is  any  analytic  study  designed 
to  assist  a  decisionmaker  identify  a  preferred  choice  from 
among  possible  alternatives.  In  a  military  context,  typical 
analyses  might  tackle  such  questions  as  the  extent  to  which 
aircraft  should  be  repaired  at  a  depot  rather  than  on  the 
base;  the  possible  characteristics  of  a  new  strategic  bomber 
and  whether  one  should  be  developed  or  not;  whether  tactical 
air  wings  or  carrier  task  forces  should  be  substituted  for 
U.S.  ground  divisions  in  NATO;  or  whether  we  should  modify 
the  test  ban  treaty  now  that  the  Chicoms  have  nuclear  weapons 
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and,  if  so,  how.  Each  such  analysis  involves  as  one  stage  a 
comparison  of  alternative  courses  of  action  in  terms  of  their 
costs  and  their  effectiveness  in  attaining  some  specific 
objective.  This  is  cost— effectiveness  analysis,  narrowly 
defined.  Usually  it  consists  of  an  attempt  to  minimize  cost 
subject  to  some  mission  requirement  (which  may  not  be  measurab] 
in  dollar  terms)  or,  conversely,  to  maximize  some  physical 
measure  of  output  subject  to  a  budget  constraint. 

Since  this  comparison  often  receives  the  lion's  share  of 
attention  by  the  participants,  by  a  natural  extension,  the 
entire  study  is  called  a  cost-effectiveness  analysis.  But 
such  emphasis  is  likely  to  be  misplaced  in  an  analysis  designe< 
to  furnish  policy  advice.  Other  facets  of  the  problem  may  be 
of  greater  significance  to  its  solution:  the  specification 
of  the  right  objectives,  the  determination  of  a  satisfactory 
way  to  measure  the  degree  of  accomplishment  of  these  objective! 
or  the  discovery  of  better  alternatives. 

Let  me  try  to  illustrate  this  last  point  with  a  homely 
example . 

Suppose  a  family  has  decided  to  buy  a  television  set. 

Not  only  is  their  objective  fairly  clear,  but,  if  they  have 
paid  due  attention  to  the  advertisements,  their  alternatives 
are  well-defined.  The  situation  is  then  or.o  for  cost— effec¬ 
tiveness  analysis,  narrowly  defined.  The  only  significant 
questions  the  family  need  answer  concern  the  differences  among 
the  available  sets  in  both  performance  and  cost.  With  a  littL 
care,  making  proper  allowance  for  financing,  depreciation, 
and  maintenance,  they  can  estimate,  say,  the  five  year  pro¬ 
curement  and  operating  cost  of  any  particular  set.  and  do  so 
with  a  feeling  that  they  are  well  inside  the  ball  park.  They 
will  discover,  of  course,  that  finding  a  standard  for  measurin 
the  effectiveness  of  the  various  sets  is  somewhat  more  difficu 
For  one  thing,  the  problem  is  multidimensional — they  must 
consider  color  quality,  the  option  for  remote  control, 
portability,  screen  size,  and  so  forth.  But,  ordinarily. 
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one  consideration — perhaps  color — dominates.  On  this  basis, 
they  can  go  look  at  some  color  sets,  compare  costs  against 
color  quality,  and  finally  determine  a  best  buy. 

Now  suppose  the  family  finds  they  have  more  money  to 
spend  and  thus  decide  to  increase  their  standard  of  living 
— a  decision  similar  tc  one  to  strengthen  the  U.S.  defense 
posture  by  increasing  the  military  budget.  This  is  a  situation 
for  a  broader  analysis.  They  first  need  to  investigate  their 
goals  or  objectives  and  look  into  the  full  range  of  alternatives 
— a  third  car,  a  piano,  a  country  club  membership.  They  then 
need  to  find  ways  to  measure  the  effectiveness  of  these  alter¬ 
natives  and  establish  criteria  for  choice  among  them.  Here, 
because  the  alternatives  are  so  dissimilar,  determining  what 
they  want  to  do  is  the  major  problem;  how  to  do  it  and  to 
determine  what  it  costs  may  become  a  comparatively  minor  one. 

To  distinguish  this  broader  analysis  from  a  narrow  cost- 
effectiveness  comparison,  I  call  the  former  a  ’’systems  analysis." 
Numerous  other  terms — operations  analysis,  operations  research, 
systems  engineering,  cost— utility  analysis — might  also  be  used, 
depending  on  the  context,  and,  to  different  people,  they  might 
imply  some  subtle  distinction.  But  they  all  convey  the  same 
general  meaning.  Moreover,  there  exists  between  them  no  dis¬ 
tinctions  in  principle.  Whatever  differences  may  be  found 
are  simply  matters  of  degree,  emphasis,  and  context.  What  is 
important  for  our  discussion,  therefore,  is  the  characteristics 
they  have  in  common.  These  include  an  effort  to  make  comparisons 
systematically  in  quantitative  terms  using  a  logical  sequence 
of  steps  that  can  be  retraced  and  verified  by  others. 

In  summary,  what  I  am  saying  is  that  to  qualify  as  a 
complete  analysis  a  study  must  look  at  the  entire  problem  and 
look  at  it  as  a  whole.  Thus,  characteristically,  such  ar, 
analysis  will  involve  a  systematic  investigation  of  the 
decisionmaker's  objectives  and  of  the  relevant  criteria;  a 
comparison— quantitative  where  possible — of  the  costs, 
effectivenesses,  and  risks  associated  with  the  alternative 
policies  or  strategies  for  achieving  each  objective;  and  an 


attempt  to  formulate  additional  alternatives  if  those  examined 
are  found  wanting.  Although  I  prefer  ’’systems  analysis”  for 
the  broader  analysis,  in  what  follows  I’ll  use  the  term  cost- 
effectiveness  for  the  full  range,  broad  and  narrow,  of  analytic 
approaches  to  problems  of  choice. 

THE  ESSENCE  OF  THE  METHOD 

What  is  there  about  an  analytic  approach  that  makes  it 
better  or  more  useful  than  other  ways  to  furnish  advice — than, 
say,  an  expert  or  a  committee?  In  areas,  such  as  defense 
planning,  where  there  is  no  accepted  theoretical  foundation, 
advice  obtained  from  experts  working  individually  or  as  a 
committee  depends  largely  on  subjective  judgment.  So  does  the 
advice  from  cost-effectiveness  analysis.  But  the  virtue  of 
analysis  is  that  it  is  able  to  make  a  more  systematic  and 
efficient  use  of  judgment  than  any  of  its  alternatives.  The 
essence  of  the  method  is  to  construct  and  operate  within  a 
’’model” — an  idealization  of  the  situation  appropriate  to  the 
problem.  Such  a  model — which  may  take  such  forms  as  a  compute! 
program,  a  war  game,  or  a  set  of  questionnaires — introduces  a 
precise  structure  and  terminology  that  serve  primarily  as  a 
means  of  communication,  enabling  the  participants  in  the  study 
to  make  their  judgments  in  a  concrete  context.  Moreover, 
through  feedback — the  results  of  computation,  the  counter¬ 
moves  in  the  war  game,  or  the  answers  to  the  questionnaires 
— the  model  helps  the  decisionmaker,  the  analysts,  and  the 
experts  on  whom  they  depend  to  revise  their  earlier  judgments 
and  thus  to  arrive  at  a  clearer  understanding  of  the  problem 
and  its  context. 

The  central  importance  of  the  model  can  be  seen  moct 
readily,  perhaps,  by  looking  at  its  relation  to  the  other 
elements  of  analysis.  There  are  five  altogether.  Each 
of  them  is  present  in  every  analysis  of  choice,  although 
they  may  not  always  be  explicitly  identified. 


1.  The  objective  (or  objectives).  Cost-effectiveness 
analysis  is  undertaken  primarily  to  halp  choose  a  policy 

or  course  of  action.  One  of  the  first  and  most  important, 
tasks  of  the  analyst  is  to  attempt  to  discover  what 
objectives  the  decisionmaker  is,  or  should  be,  trying  to 
attain  through  thi«  policy,  and  how  to  measure  the  extent 
to  which  they  are,  in  fact,  attained.  This  done,  strategies, 
forces,  or  equipment  are  examined,  compared,  and  chosen  on 
the  basis  of  how  well  and  how  cheaply  they  can  accomplish 
these  objectives. 

2.  The  alternatives.  The  alternatives  are  the  means 
by  which  it  is  hoped  the  objectives  can  be  attained.  They 
need  not  be  obvious  substitutes  for  one  another  or  perform 
the  same  specific  function.  Thus,  to  protect  civilians 
against  air  attack,  shelters,  "shooting”  defenses,  counter- 
force  attack,  and  retaliatory  striking  power  are  all 
alternatives. 

3.  The  costs.  The  choice  of  a  particular  alternative 
for  accomplishing  the  objectives  implies  that  certain 
specific  resources  can  no  longer  be  used  for  other  purposes. 
These  are  the  costs.  In  analyses  for  a  future  time  period, 
most  costs  can  be  measured  in  money,  but  their  true  measure 
is  in  terms  of  the  opportunities  that  they  preclude.  Thus, 
if  we  are  comparing  ways  to  eliminate  guerrillas,  the 
damage  to  nonparticipating  civilians  caused  by  the  various 
alternatives  must  be  considered  a  cost,  for  such  damage 

may  recruit  more  guerrillas. 

4.  A  model  (or  models).  A  model  is  a  representation 
of  reality  which  abstracts  the  features  of  the  situation 
relevant  to  the  question  being  studied.  The  means  of 
representation  may  vary  from  a  set  of  mathematical  equations 
or  a  computer  program  to  a  purely  verbal  description  of 

the  situation,  in  which  judgment  alone  is  used  to  assess 
the  consequences  of  various  choices.  In  cost-effectiveness 
analysis  (or  any  analysis  of  choice),  the  role  of  the  model 


(or  models,  for  it  may  be  inappropriate  or  absurd  to 
attempt  to  incorporate  all  the  aspects  of  a  problem  in  a 
single  formulation)  is  to  predict  the  costs  that  each 
alternative  would  incur  and  the  extent  to  which  each 
alternative  would  assist  in  attaining  the  objectives. 

5.  A  criterion.  A  criterion  is  a  rule  or  standard 
by  which  to  rank  the  alternatives  in  order  of  desirability 
and  choose  the  most  promising.  It  provides  a  means  for 
weighing  cost  against  effectiveness. 

The  relation  between  these  elements  is  illustrated 
by  the  chart  on  the  opposite  page. 

The  process  of  analysis  may  be  outlined  as  follows. 

The  consequences  of  choosing  an  alternative  (which  may 
have  to  be  discovered  or  invented  as  part  of  the  analysis) 
are  obtained  by  means  of  the  model  or  a  series  of  models. 
These  consequences  tell  us  how  effective  each  particular 
alternative  is  in  the  attainment  of  the  obj ectives  (which 
requires  that  we  have  a  measure  of  effectiveness  for  each 
objective)  and  what  the  costs  are.  A  criterion  can  then 
be  used  to  arrange  the  alternatives  in  order  of  preference. 

Unfortunately,  things  are  seldom  so  tidy:  alternatives 
are  not  adequate  to  attain  the  objectives;  the  measures 
of  effectiveness  do  not  really  measure  the  extent  to  which 
the  objectives  are  attained;  the  predictions  from  the  model 
are  full  of  uncertainties;  and  other  criteria  which  look 
almost  as  attractive  as  the  one  chosen  may  lead  to  a 
different  order  of  preference. 

THE  VIRTUES 

In  stating  the  purpose  of  cost-effectiveness  analysis, 
we  have,  in  a  sense,  said  what  it  can  do.  It  can  be 
applied  to  a  range  of  problems  extending  from  the  very 
narrow  to  the  very  broad.  At  one  end  of  the  range,  it 
may  be  applied  to  determining  how  much  of  the  Air  Force 
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construction  budget  allocated  to  hangars  should  be 
assigned  to  a  given  base,  or  what  type  of  all-weather 
landing  system  should  be  installed  in  a  new  transport 
aircraft,  or  whether  on  a  particular  base  the  electrical 
maintenance  shop  should  be  unified  with  some  other  shop. 

At  the  other  end,  it  can  help  to  decide  how  much  should 
be  spent  on  national  defense,  or  how  the  defense  budget 
should  be  allocated  between  strategic  and  general  purpose 
forces,  or  whether  the  additional  capability  provided  by 
another  carrier  task  force  would  be  worth  the  cost.  In 
all  these  cases,  competent  cost-effectiveness  analysis 
can  help  a  decisionmaker  to  understand  the  relevant1 
alternatives  and  the  key  interactions  by  giving  him  an 
estimate  of  the  costs,  risks,  and  possible  payoffs 
associated  , ith  each  course  of  action.  In  so  doing,  it  may 
sharpen  his  intuition;  it  will  certainly  broaden  his 
basis  for  judgment.  In  practically  no  cases,  however, 
should  the  analyst  expect  to  demonstrata  to  a  decisionmaker 
beyond  all  reasonable  doubt  that  he  should  choose  a 
particular  course  of  action. 

It  is  easy,  unfortunately,  to  exaggerate  the  degree 
of  assistance  that  analysis  can  offer  a  policymaker. 

Using  value  judgments,  imprecise  knowledge,  and  Intuitive 
estimates  of  enemy  intent,  gleaned  from  specialists  or 
from  the  policymaker  himself,  a  study  can  do  little  more 
than  assess  some  of  the  implications  of  choosing  one 
alternative  over  another.  This  almost  always  helps  the 
decisionmaker  make  a  better  decision  than  he  would 
otherwise  make,  but  the  man  who  has  the  responsibility 
must  interpret  such  assessments  in  the  light  of  his  own 
knowledge,  values,  and  estimates,  and  assess  other 
implications  himself.  The  decision  must  become  his  own. 
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THE  DEFECTS 

Every  cost-effectiveness  analysis  has  defects.  Some 
of  these  are  limitations  inherent  in  all  analysis  of  choice. 
Others  are  flaws  or  errors  which,  hopefully,  will  disappear 
as  we  learn  to  do  better,  more  thorough,  and  complete  analyses. 

Most  flaws  are  caused  by  such  pitfalls  as  emphasis 
on  working  with  the  model  instead  of  the  question,  or 
concentration  on  the  type  of  uncertainty  that  can  be 
treated  analytically  by  Monte  Carlo  or  other  statistical 
techniques  rather  than  on  the  real  uncertainties,  or 
neglect  of  the  subjective  elements  in  the  analysis. 

I'll  elaborate  on  what  I  consider  the  most  dangerous 
pitfall  or  source  of  defects.  This  is  the  attention  bias. 

It  is  frequently  caused  by  a  cherished  belief  or  an 
unconscious  adherence  to  c  "party  line."  All  organizations 
foster  one  to  some  extent;  RAND,  the  military  services,  and 
the  DOD  are  no  exception.  My  feeling  is  that  Herman  Kahn 
was  right  when  he  called  the  party  line  "the  most  important 
single  reason  for  the  tremendous  miscalculations  that  are 
made  in  foreseeing  and  preparing  for  technical  advances  or 
changes  in  the  strategic  situation."  Examples  of  an 
attention  bias  are  plentiful — the  military  planner  whose 
gaze  is  so  fixed  on  "winning"  local  wars  that  he  excludes 
other  considerations,  or  so  fixed  on  maximizing  deterrence 
that  he  completely  disregards  what  might  happen  should 
deterrence  fail;  the  weaponeer  who  is  so  fascinated  by 
startling  new  weapons  that  he  assumes  they  will  of  course 
be  used;  the  State  Department  negotiator  who  seeks  to 
conciliate  the  potential  enemy  at  a  military  cost  that  is 
far  too  great,  because  he  is  unaware  of  it.  In  fact,  it  is 
this  failure  to  realize  the  vital  interdependence  among 

- 1 - 

H.  Kahn  and  I.  Mann,  Ten  Common  Pitfalls,  The  RAND 
Corporation,  RM-1937,  July  17,  1957,  p.  42. 
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political  purpose,  diplomacy,  military  posture,  economics, 
and  technical  feasibility  that  is  the  typical  flaw  in 
most  practitioners'  approach  to  national  security  analysis. 

Pitfalls  are  one  thing,  and  the  inherent  limitations  of 
analysis  itself  another.  It  is  these  limitations  that  confin 
analysis  to  an  advisory  role.  I  single  out  four  for 
further  comment:  analysis  is  necessarily  incomplete; 
measures  of  effectiveness  are  inevitably  approximate; 
ways  to  predict  the  future  are  lacking;  and  all  analysis' 
of  choice  falls  short  of  scientific  research. 

Analysis  is  necessarily  incomplete 

Time  and  money  costs  obviously  place  sharp  limits 
on  how  far  any  inquiry  can  be  carried.  The  very  fact 
that  time  moves  on  means  that  a  correct  choice  at  a 
given  time  may  soon  be  outdated  by  events  and  that  goals 
set  down  at  the  start  may  not  be  final.  This  is 
particularly  important  in  military  analysis,  for  the 
decisionmaker  can  wait  only  so  long  for  an  answer.  Other 
costs  are  important  here  too.  For  instance,  we  would 
like  to  find  out  what  the  Chicoms  would  do  if  we  put  an 
end  to  all  military  aid  to  Southeast  Asia.  One  way  to 
get  this  information  would  be  to  stop  such  aid.  But 
while  this  would  clearly  be  cheap  in  dollars,  the 
likelihood  of  other  costs  precludes  at  once  this  type 
of  investigation. 

Still  more  important,  however,  is  the  general  fact 
that,  even  with  no  limitations  of  time  and  money,  analysis 
can  never  treat  all  the  considerations  that  may  be  relevant. 
Some  are  to'j  intangible.  For  example,  such  qualities  of 
a  system  as  its  flexibility,  its  compatibility  with  other 
systems  (including  some  that  are  yet  to  be  developed), 
its  contributions  to  national  prestige  abroad,  and  its 
impact  on  domestic  political  constraints  can,  and  possibly 
should,  play  as  important  a  role  in  the  choice  of  alternativ 
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force  postures  as  any  idealized  war  outcome  calculations. 

Ways  to  measure  these  things  even  approximately  don't 
exist  today  and  they  must  be  handled  subjectively.  (And 
if  we  find  out  how  to  measure  these  things,  other 
political,  psychological,  and  sociological  intangibles 
will  still  be  left.)  The  analyst  can  apply  his  own 
judgment  and  intuition  to  these  considerations  and  thus 
make  them  part  of  the  study,  but  the  decisionmaker  will  rightly 
insist  on  applying  his  own. 


Measures  of  effectiveness  are  approximate 


In  military  cost— effectiveness  analysis,  measures 
of  effectiveness  are  at  best  reasonably  satisfactory 
approximations  for  measuring  such  vaguely  defined  objectives 
as  deterrence  or  victory.  Sometimes  the  best  that  can  be 
done  is  to  find  measures  which  point  in  the  right  direction. 
Consider  deterrence,  for  instance.  It  exists  only  in  the 
mind — and  in  the  enemy's  mind  at  that.  We  cannot,  therefore, 
measure  directly  the  effectiveness  of  alternatives  we 
hope  will  lead  to  deterrence,  but  must  use  instead  such 
approximations  as  the  potential  mortalities  that  we  might 
inflict  or  the  square  feet  of  roof  cover  we  might  destroy. 
Consequently,  even  if  a  comparison  of  two  systems  indicated 
that  one  could  inflict  50  per  cent  more  casualties  on  the 
enemy  than  the  other,  we  could  not  conclude  that  this 
means  the  system  supplies  50  per  cent  more  deterrence. 

In  fact,  in  some  circumstances,  we  find  arguments  that 
the  system  which  threatens  the  greatest  number  of  casualties 
may  provide  the  least  deterrence.' 

Moreover,  we  can't  be  as  confident  about  the  accuracy 
of  our  estimates  of  effectiveness  as  we  are  about  our 


cost  estimates.  It  is  the  opinion  of  one  analyst  who 
has  studied  che  problem  of  estimating  casualties  that 
if  a  pre— World  War  II  estimator  had  worked  analogously 


-12- 


to  his  brother  of  today,  had  known  his  trade  exceptionally 
well,  had  been  knowledgeable  about  the  means  by  which 
World  War  II  military  actions  produced  casualties,  had 
known  the  probabilities  associated  with  each  weapon,  and 
could  have  estimated  the  number  of  people  subject  to  each 
weapon — then  such  an  estimator  would  have  underestimated 
the  total  cost  in  human  lives  of  the  war  to  the  Soviets 
by  a  factor  of  between  three  and  four. 

Such  an  error  in  the  measurement  of  effectiveness 
may  not  be  too  important  if  we  are  comparing  two  systems 
that  are  not  radically  unlike  one  another — two  ground 
attack  aircraft,  say.  But  at  higher  levels  of  optimi¬ 
zation — tanks  vs.  aircraft  or  missiles — gross  differences 
in  system  effectiveness  may  be  obscured  by  gross 
differences  in  the  quality  of  damage  assessment. 

In  summary,  what  I  am  saying  is  that  we  don't  know 
how  to  translate  a  capability  to  create  casualties  (as 
perceived  by  the  enemy)  into  deterrence,  we  don't  know 
how  they  will  compute  the  casualty— producing  capability  of  01 
forces,  and  we  don't  even  know  how  to  do  it  ourselves 
very  accurately.  This  is  for  a  hard  measure  of  effective¬ 
ness — the  soft  ones  are  worse.' 

Don't  misunderstand  me — the  determination  of  even 
the  dollar  costs  of  a  military  action  is  not  simple.  The 
cost  of  a  change  in  force  posture,  say,  is  measured  by  a 
price  that  typically  includes  the  costs  of  R&D,  initial 
investment,  and  annual  operation,  and  these  costs  will 
have  to  cover  the  various  weapons,  equipment,  and  vehicles 
involved,  and  also  the  whole  materiel  and  manpower 
structure  that  underlies  the  entire  lifetime  of  the  system. 
Consequently,  it  requires  a  great  deal  of  sophistication 
to  learn  how  to  cost  a  force  posture — to  learn  what  things  go 
into  making  up  a  posture  and  how  their  costs  can  be  found. 
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It  takes  even  more  know-how  and  research  to  estimate  the 
costs  of  weapons  and  forces  that  are  as  yet  only  concepts. 
But  with  care  and  experience,  once  we  decide  what  we  are 
costing,  we  can  do  fairly  well. 

No  satisfactory  way  to  predict  the  future  exists 

While  it  is  possible  to  forecast  events  to  come  in 
the  sense  of  mapping  out  possible  futures,  there  is  no 
satisfactory  way  to  predict  a  single  future  in  terms  of 
which  we  can  work  out  the  best  system  or  determine  an 
optimum  policy.  Consequently,  we  must  consider  a  range 
of  possible  futures  or  contingencies.  In  any  one  of  these 
we  may  be  able  to  designate  a  preferred  course  of  action, 
but  we  have  no  way  to  determine  one  for  the  entire  range 
of  possibilities.  We  can  design  a  force  structure  for 
a  particular  war  in  a  particular  place,  but  we  have  no 
surefire  way  to  work  out  a  structure  that  is  good  for 
the  entire  spectrum  of  future  wars  in  all  the  places  they 
may  occur. 

Consequently,  defense  planning  is  rich  in  the  kind 
of  analysis  that  tells  what  damage  could  be  done  to  the 
United  States  given  a  particular  enemy  force  structure 
(or,  to  put  it  another  way,  what  enemy  requirements  would 
be  to  achieve  a  given  destruction);  but  it  is  poor  in 
the  kinds  of  analyses  that  evaluate  how  we  will  actually 
stand  in  relation  to  the  Soviets  in  years  to  come. 

All  analysis  of  choice  falls  short  of  scientific  research 

No  matter  how  we  strive  to  maintain  standards  of 
scientific  inquiry  or  how  closely  we  attempt  to  follow 
scientific  methods,  we  cannot  turn  cost-effectiveness 
analysis  into  science.  Its  objective,  in  contrast  to  that 
of  science,  is  primarily  jo  recommend — or  at  least  to 
suggest — policy,  rather  than  merely  to  understand  and 


predict.  Like  engineering,  it  seeks  to  use  the  results 
of  science  to  do  things  well  and  cheaply.  Yet  it  differs 
from  ordinary  engineering  in  its  enormous  responsibility, 
in  sometimes  being  forced  by  the  nature  or  urgency  of  a 
problem  to  substitute  intuition  for  verifiable  knowledge, 
in  the  unusual  difficulty  of  appraising— or  even  discoverinj 
— a  value  system  applicable  to  its  problems,  and  in  the 
absence  of  ways  to  test  its  validity. 

Except  for  this  inability  to  verify,  cost— ef fectivenes.* 
analysis  may  still  look  like  a  purely  rational  approach  to 
decisionmaking,  a  coldly  objective,  scientific  method  free 
of  the  human  attributes  of  preconceived  ideas  and  partisan 
bias  and  judgment  and  intuition. 

It  isn't,  really.  Human  judgment  is  used  in  designing 
the  analysis;  in  deciding  what  alternatives  to  consider, 
what  factors  are  relevant,  what  the  interrelations  between 
these  factors  are,  and  what  numerical  values  to  choose; 
and  in  analyzing  and  interpreting  the  results  of  the 
analysis.  This  fact — that  judgment  and  intuition  permeate 
all  analysis — should  be  remembered  when  we  examine  the 
results  that  come,  with  apparent  high  precision,  from 
analysis . 

THE  FUTURE 

And  finally,  what  of  the  future?  Where  they  can  be 
obtained,  quantitative  estimates  of  costs  and  effectiveness 
are  clearly  helpful  to  any  intelligent  discussion  of 
national  security.  In  current  Department  of  Defense 
practice  those  quantitative  estimates  are  obtained  by 
means  of  cost--ef f ectiveness  analyses,  sometimes  assisted 
by  computerized  games.  Many  people,  however,  and  some 
of  you  are  probably  among  them,  are  vaguely  uneasy  about 
the  p*tMcuiar  way  these  estimates  are  made  and  their 
incieasingiy  important  role  in  military  planning. 
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For  example,  an  Air  Force  officer  writes  that 

computer— oriented  planning  techniques  are  dangerous;  that 

mathematical  models  of  future  wars  are  inadequate  for 

defense  planning;  and  that  scientific  methods  cannot 

handle  those  acts  of  will  which  determine  the  conduct  of 

war.  A  Congressman  says,  "We  should  not  allow  cost- 

effectiveness  to  cost  us  our  effectiveness  in  national 

#  #  # 

security  matters.”  A  Senator  remarks,  "Our  potential 
enemies  may  not  use  the  same  cost-effectiveness  criteria 
and  thus  oppose  us  with  the  best  weapons  their  technology 
can  provide.  This  would  create  an  intolerable  peril  to 
the  national  security,” 

Some  of  this  skepticism  may  be  justified,  for  the  work 
may  not  always  be  competently  done  or  used  with  its  limita¬ 
tions  in  mind.  But  those  critics  who  hold  that  national 
defense  decisions  are  being  made  today  solely  by  consideration 
of  computer  calculations  are  not  only  premature  in  their 
belief  (to  say  the  least),  but  probably  have  a  basic  mis¬ 
understanding  of  how  such  calculations  must,  in  fact,  always 
be  used.  As  you  should  know,  this  process  is  today  rampant 
with  dogma,  service  rivalries,  special  interests,  and  horse¬ 
trading — so  much  so  that,  in  the  opinion  of  some  analysts,  a 
computerized  solution  based  on  a  relaxation  of  these  human 
constraints  might  lead  to  something  better.  But  this  is 
wishful  thinking.  As  we  have  pointed  out,  analysis  has 
limits.  Broad  problems  always  involve  considerations  which 
require  reliance  on  judgment,  intuition,  and  experience. 

And  incidentally,  the  interservice  rivalries  and 

Colonel  Francis  X.  Kane,  USAF,  Security  is  Too 
Important  to  be  Left  to  Computers,"  Fortune  Magazine, 

April  1964. 

*  # 

Representative  Laird  of  Wisconsin,  Missile/ Space 

Daily,  April  7,  1964,  p.  161. 

*** 

Senator  John  0.  Pastore  of  Rhode  Island,  U.S.  News 
and  World  Report,  January  6,  1964. 
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bargaining  are  not  all  wasteful;  they  have  the  virtue 
of  insuring  that  the  work  is  carefully  scrutinized  and 
that  alternatives  and  considerations  to  which  we  are 
blind  through  our  cherished  beliefs  are  brought  to  our 
attention. 

Resistance  by  the  military  to  the  use  of  cost- 
effectiveness  or  systems  analysis  to  help  in  broad  problems 
of  strategy  is  gradually  breaking  down.  Military  planning 
and  strategy  have  always  involved  more  art  than  science; 
what  is  happening  is  that  the  art  form  is  changing  from 
an  ad  hoc,  seat— of— the— pants  approach  based  on  intuition 
and  experience  to  one  based  on  analysis  supported  by 
intuition  and  experience.  Military  planning  itself  has 
changed.  With  this  change  the  computer  is  becoming 
increasingly  significant — as  an  automaton ,  a  process 
controller,  a  service  trouble-shooting  technician,  a 
complex  information  processor,  and  a  decision  aid .  Its 
usefulness  in  serving  these  ends  can  be  expected  to  grow. 

But  at  the  same  time,  it  is  important  to  note  that  even 
the  best  computer  is  no  more  than  a  tool  to  expedite 
analysis.  Even  in  the  narrowest  military  decisions, 
considerations  not  subject  to  any  sort  of  quantitative 
analysis  can  always  be  present.  Big  decisions  cannot  be 
the  automatic  consequence  of  a  computer  program,  of  cost- 
effectiveness  analysis,  operations  research,  or  any 
application  of  mathematical  models. 

For  broad  studies,  involving  force  posture  and  composi¬ 
tion  or  the  strategy  to  achieve  foreign  policy  objectives, 
intuitive,  subjective,  even  ad  hoc  study  schemes  must  contim 
to  be  used — but  supplemented  to  an  increasing  extent  by  cost- 
effectiveness  analysis.  And  as  ingredients  of  this  analysis 
in  recognition  of  the  need  for  judgment,  a  greater  use  of 
scenarios,  gaming,  and  other  techniques,  such  as  questionnai: 
for  the  systematic  employment  of  experts  can  be  expected, 
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along  with  an  increasing  use  of  computer-based  analysis 
for  problems  where  it  is  appropriate. 

Moreover,  the  scope  will  broaden.  Cost-effectiveness 
analyses  must  enter  the  domain  of  the  social  sciences. 
Here,  as  Olaf  Helmer  remarks  in  his  perceptive  essay. 
Social  Technology: 

we  are  faced  with  an  abundance  of  challenges: 
how  to  keep  the  peace,  how  to  alleviate  the 
hardships  of  social  change,  how  to  provide 
food  and  comfort  for  the  inaffluent,  how  to 
improve  the  social  institutions  and  the  values 
of  the  affluent,  how  to  cope  with  revolutionary 
innovations,  and  so  on.* 

Some  of  you  evidently  believe  that  cost— effectiveness 
analysis  and  related  techniques  can  help  with  these 
problems,  for  later  in  this  Symposium  we  will  hear  of 
applications  to  welfare  and  other  government  domestic 
programs.  But  to  go  very  far  into  the  domain  of  the 
social  sciences,  new  techniques  need  to  be  developed. 
Indeed,  as  Helmer  further  remarks: 

many  of  the  difficulties  which  beset  our  world 
today  can  be  explained  by  the  fact  that  progress 
in  the  social— science  domain  has  lagged  far 
behind  that  in  the  physical  sciences.** 

Now,  while  many  people  are  extremely  pessimistic 
about  the  possibilities  of  finding  any  techniques  that 
will  be  really  helpful,  it  is  the  thesis  of  Dr.  Helmer 
that  a  completely  pessimistic  outlook  is  unwarranted: 

The  potential  reward  from  a  reorientation 
of  some  of  the  effort  in  the  social— science 
area  toward  social  technology,  employing 
operations— research  techniques,  is  considerable; 
it  may  even  equal  or  exceed  in  importance  that 
of  the  achievements  which  are  credited  to  the 
technologies  arising  out  of  the  physical  sciences. 


- * 

Olaf  Helmer,  Social  Technology,  The  RAND  Corporation, 
P—3063.  Paper  presented  at  the  Futuribles  Conference  in 
Paris,  April  1965. 

** 
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The  methodological  implications  of  such  a 
reorientation  can  be  simply  summarized  under 
the  headings  of  "operational  model  building" 
and  "systematic  use  of  expertise".* 

The  first  of  these  approaches  is  the  essence  of 
current  cost-effectiveness  analysis.  The  second  is  a 
necessary  ingredient  of  the  first  that  has  just  lately 
become  the  subject  of  research. 

CONCLUDING  REMARKS 

How  can  we  summarize  any  danger  that  there  might  be 
in  reliance  on  cost-effectiveness  analysis,  or  on  any 
similar  approach  to  broad  government  decisions?  First, 
since  many  factors  fundamental  to  problems  of  national 
welfare  are  not  subject  to  rigorous,  quantitative  analysis, 
they  may  possibly  be  neglected,  deliberately  set  aside, 
or  improperly  weighted  in  the  analysis  itself  or  in  the 
decision  based  on  such  analysis.  Second,  an  analysis 
may,  on  the  surface,  appear  so  scientific  and  quantitative 
that  it  may  be  assigned  a  validity  not  justified  by  the 
many  subjective  judgments  involved.  In  other  words,  we 
may  be  so  mesmerized  by  the  beauty  and  precision  of  the 
numbers  that  we  overlook  the  simplifications  made  to 
achieve  this  precision,  neglect  analysis  of  the  qualitative 
factors,  and  overemphasize  the  importance  of  idealized 
calculations  to  the  decision  process.  But  better  analysis 
and  careful  attention  to  where  analysis  ends  and  judgment 
begins  should  help  to  reduce  these  dangers. 

At  the  very  least,  cost-effectiveness  analysis  offers 
a  way  to  choose  the  numerical  quantities  related  to  a 
weapon  system  so  that  they  are  logically  consistent  with 
each  other,  with  an  assumed  objective,  and  with  the 
calculator's  expectation  of  the  future.  The  method 
provides  its  answers  by  processes  that  are  accessible 

*Ibid . 
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to  critical  examination,  capable  of  duplication  by  others, 
and,  more  or  less,  readily  modified  as  new  information 
becomes  available.  And,  in  contrast  to  other  aids  to 
decisionmaking,  which  share  the  same  limitations,  it 
extracts  everything  possible  from  scientific  methods, 
and  its  virtues  are  the  virtues  of  those  methods.  But, 
even  within  the  Department  of  Defense,  its  capabilities 
have  as  yet  to  be  fully  exploited. 


